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ABSTRACT 

This paper describes seven studies (all that could be 
found on the subject) comparing nine respondent selection procedures 
with regard to heir potential to reduce refusal rates and to 
increase representativeness of samples; particular concerns are 
underrepresentation of men and multiple-adult households. The paper 
states that (1) the seven studies compare two or more procedures; (2) 
they have been published or presented to professional meetings; (3) 
they are described in terms of their historical evolution as 
improvements on the Kish procedure for telephone surveying; and (4) 
comparisons show that all the methods vary in their biases. The paper 
suggests that the last-birthday, Collier-Hagan, and Bryant 
modification of the Troldahl-Carter method appear to be the most 
representative procedures and that Ki3h and Troldahl-Carter 
techniques appear to lead to less representative samples than other 
techniques. Other procedures reported in the paper include two 
non-probability ones, a **Stanford modification** of Troldahl-Carter, 
and a modification of the Troldahl-Carter-Bryant correction for 
O'.^r representation of women. The paper suggests more systematic 
testing of different combinations of respondent selection methods and 
more replication are needed to learn more abou'. external validity, 
representativeness, and other criteria to help evaluate the most 
effective selection strategies for specific projects. (Three tables 
of data and six notes are included, and 26 references are appended.) 
(MS) 



******************************* 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 
*********************************************************************** 



ERIC 



VARIATIONS IN SURVEY RESPONDENT SELECTION BY TELEPHONE 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



By 

Cecilie Gaziano 

Market Research Consultant 
Minneapolis, Minnesota 



U.S DEPARTMENT OF EDUCmTJON 

Off»ce o< EducatJOAai Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 

CENTER (ERIC) 
r This <Jocoment has been reproduced as 

received Irom the person or organization 

Originatir>g it 
e: Mino' Changes have been made to improve 

reproduction quality 

• Points ot view or opmtons statad m this docu- 
ment do not necessarily represent ot<«cial 
OERJ position or po»»cy 



MAILING ADDRESS: 

4511 Fremont Avenue South 
Minneapolis, MN 55409 
Phone (612) 825-8887 



Presented to the 43rd Annual Conference 
of the American Association for Public Opinion Research in Toronto. Canada. 



MAY 21. 1988 



2 



% 



Cecilie Gaziano 

Session on "How Survey Design Decisions Affect Non-response." 
9 a.m. to 10:30 a.m., Saturday, May 21. 1988. 
AAPOR/WAPOR Conference. Toronto. Canada. 

ABSTRACT: 

VARIATIONS IN SURVEY RESPONDENT SELECTION BY TELEPHONE 

Seven studies comparing nine respondent selection procedures are described 
in this paper with regard to their potential to reduce refusal rates and to 
increase representativeness of samples. Particular concerns are 
under representation of men and multiple-adult households. These seven studies 
are all of the studies which could be found on this topic, which compare two or 
more procedures and which have been published or presented to professional 
meetings. The paper describes them in terms of their historical evolution as 
improvements on the Kish procedure for telephone surveying. Interest in such 
modifications has increased in recent years. 

All the methods vary in their biases. The last-birthday. Collier-Hagan. 
and Bryant modification of the Troldahl-Carter method appear to be the most 
representative procedures. Kish and Troldahl-Carter techniques appear to lead 
to less representative samples than other techniques. Other procedures reported 
include two non-probability ones, a "Stanford modification" of Troldahl-Carter. 
and a modification of the Troldahl-Carter-Pryant correction for over- 
representation of women. 

More systematic testing of different combinations of respondent selection 
methods and more replication are needed to learn more about external validity, 
representativeness, and other criteria to help evaluate the most effective 
selection strategies for specific projects. What we now know about respondent 
selection methods is disappointingly little. 



VARIATIONS IN SURVEY RESPONDENT SELECTION BY TELEPHONE 
Telephone survey selection of adults within households typically presents 
several problems. One main one is how to determine selection of the appropriate 
individual within the household to be interviewed. Second, while respondents 
are being selected, major efforts are required to minimize the refusal rate and 
simultaneously tc maximize representativeness of the sample. 

This paper will describe seven studies which compare two or more respondent 
selection procedures. These seven studies are all of the reports which 
could be found on this topic, which compare two or more techniques and which 
have been published or presented to professional meetings. They will be 
discussed in order of their historical evolution as improvements on the Klsh 
procedure for telephone surveying. Most comparative studies were conducted 
relatively recently, which suggests that interest in such research has increased. 

The relative strengths and weaknesses which such research has revealed in 
these methods is evaluated, and future research directions are suggested. 
Not enough Is known about the representativeness of each method. This paper 
will focus primarily on random respondent selection by telephone. Only 
comparisons of telephone procedures will be described because no comparisons 
of procedures for in-person surveys have been uncovered in an extensive 
literature search. 

Some evidence indicates people who refuse or resist surveys tend to be 
older, to be among lower socioeconomic groups, and to live in urban areas 
(Nelson, 1982; Lavrakas, 1987). On the other hand, people difficult to 
reach have somewhat different characteristics (O'Neil. 1979). They tend to 
be younger, to be better educated and more affluent, to have white collar 
jobs, and also to live in urban areas. 

The optimum method of selecting respondents within households would 
minimize the refusal rate and improve repress itativeness bv including oeoole who 



tend to be refusers or resisters since all Individuals in their kinds of 
household do not have an equal chance of being selected. It is then necessary 
to call these households back at least three times. Some evidence indicates 
generally that the first call nets only about 25% of people in a sample, and the 
second call nets another 25%. but the point of diminishing returns is reached 
with the fourth call to a household (Dunkelberg and Day, 1973). Increasing 
refusal rates compound these problems (Steeh 1981). 

Improving representativeness becomes an especially important issue when the 
research topic is related to characteristics of the sample which are affected by 
nonresponse and lack of representativeness. One of the best arguments for the 
need to overcome such problems is Traugott's (1987) comparison of several national 
surveys showing variations in relationships between political party identifiers 
an^ candidate preference. These variations were linKed to respondent selection 
method (enumeration of individuals in households and subsampling designated 
respondents), as well as call-back persistence and full-shift Interviewing (as 
opposed to evenings only). Number of call-backs affected the proportions of 
different kinds of groups represented, as well as accuracy of results. 

The following methods of selecting adult respondents within 
households by telephone were compared among the seven studies (Table 1 shows 
which studies used which methods): 

1. Interview anyone who answers the phone ~ that is. no selection, as well 
as a variant on this, alternating males and females. 

2. Another non-probability method, asking for "the head of the household'* 
or asking alternately for male and female heads of household. 

3. The Kish procedure, developed for personal interviewing (1949). 

4. The Troldahl and Carter (1964) modification of Kish for telephone surveys. 

5. The computerized "Stanford modification." of Troldahl-Carter developed 
by Paisley and Parker (1965) to correct for a small non-random bias. 

6. Bryant's (1975) modification of the Troldahl-Carter method, correcting 
for over-reoresentation of women. 



7. The Salmon and Nichols (1980. 1983) birthday method. 

8. A modification of the Bryant version of Trolc ^hl-Carter , correcting for 
overrepresentation of women, developed by Czaja. Blair, and Sebestik (1982) 

9. An "alternative non-invasive" modification of Troldahl-Carter and 
Troldahl -Carter-Bryant versions, introduced by Hagan and Collier (1982. 1983). 

Non-random Methods 

Lavrakas (1987) has noted that non-random, or uncontrolled, respondent 
selection lacks repiesentativ* less because the most cooperative and the most 
available people will be interviewed, yielding a mix which can change from one 
survey to another. This mix is neither scientific nor high in external 
validity. Lavrakas also hoped that an increasingly knowledgeable public would 
come to expect respondent selection procedures in surveys and would believe 
non-probability methods are unprofessional. 

Surveys which select heads of household run the risk of disproportionate 
representation of women, especially in urban areas where there are more women- 
headed single-parent families (Lavrakas, 1987). If this method is chosen, the 
researcher should be aware of the area's demographic make-up and keep daily 
tallies by sex in order to ask for more males if female representation becomes 
too high (a violation of random sampling to correct for a particular bias). 

The Kish Procedure 

A very rigorous and widely used method is the Kish (1949) procedure, 
developed during a time when most interviews were in person. The interviewer 
lists by name all men in the household and their relationships to others in 
order of decreasing age. then similarly lists all women. Asking afe is not 
necessary in one-adult households or in households with two related adults. The 
informant's answers are used to choose the appropriate respondent among six (or 
eight) random number tables with rotations of possible combinations of sex and 
age. The Kish method is considered to be the most "pure." because it is the 



only well-known one which allows all household members to have an equal chance 
of being chosen. 

One principal criticism, however, is that Kish*s method requires so 
much time for complete enumeration at the beginning of interviews that 
respondents* suspicions of these questions increase the refusal rate 
(Paisley and Parker, 1965). Rapport between interviewer and respondent is 
more difficult to establish than by shorter methods, interviews are 
frequently too lengthy, and the complexity of the procedurp sometimes leads 
interviewers to introduce their own ad libbed modifications, which inhibit 
comparability. 

Further, the Kish method appears to lead to refusals early in the 
interview before substantive questions can be asked {Dillman, Gal legos, and 
Frey, 1976). This was recognized as a special problem for telephone 
surveys, because rapport between interviewer and resnondent is more tenuous 
and more easily ended at the outset than in face-to-face interviews 
(Troldahl and Carter, 1964). Tioldahl and Carter also suspected the Kish 
technique gave interviewers too much time for writing and for ?ubstituting 
their own selection procedures, and therefore, too much room for error. 
Experienced and skilled interviewers Kay, however, compensate for the 
intrusiveness of Kish questions (Lavrakas. 1987). The Kish procedure is 
less complex and problematic for one- and two-person households than it is 
for larger households. 

The Troldahl-Carter Method 

Troldahl and Carter modified Kish's procedure, primarily to meet the needs 
of their 1960 telephone survey in Minneapolis. St. Paul, and surrounding 
suburbs. Their technique also was verified by work in Champaign-Urbana . 
Illinois (Parker, 1961). The number of questions was reduced to two: 1) "How 
manv neoole 18 years or older live in vour household, counting yourself?" and 2) 
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"How many of them are men?" (p. 72). The interviewer had just four selection 
matrices from which to choose respondents. 

One disadvantage to this approach is the matrices allow selection only of 
the oldest or youngest man or woman. Adults in households with more than two 
adults whose ages fall in between the oldest and the youngest have no chance of 
being selected. Troldahl and Carter believed this violation of random sampling 
theory was quite minor, affecting on.'y two to three percent of a sample. (Kish 
estimated that one percent of selected households would include more than "ive 
adults in the late 1940*s.) 

The Paisley-Parker "Stanford Modification" 

Paisley and Parker (1965) thought the error rate was five percent in the 

Troldahl-Carter version, and they offered a solution. Whereas Troldahl and 

Carter needed only four tables. T^bat Paisley and Parker called the "Stanford 

Modification" required a vastly increased number of tables (60) to satisfy a 

neces.sary condition of maintaining equal selection probabilities for households 

of any number of adults. They described this necessary condition as follows: 

It is that number which allows all family sizes to 'come out 
even.' It is that number which allows a 3-adult family to be 
completely enumerated x times, with no persons left over, which allows 
a 4-adult family to be completely enumerated ^ times, and which allows 
a 5-adult family to be completely enumerated z times. The required 
number, therefore, is che least common denominator of 3. 4. and 5. or 
60. A 3-adult family cycles 20 times in 60 versions, a 4-adult family 
15 times, and a 5-advlt family 12 times. No smaller number of 
versions will allow all family sizes to 'come out even.' (p. 433) 

Until that time, no technique had jeen developed which allowed for 
households larger than six adults. Computers were seen as the best method of 
dealing with prodigious numbers of tables, given the considerable financial end 
time costs of paper duplication methods. 

The Bryant Correction for Too Many Females 

Bryant (1975) proposed a quite different modification of the widely-used 
Troldahl-Carter (T-C) method. Bv the 1970 census the orooortion nf 
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households headed by males had decreased to 71.6% from 77.2% In 1960. Both the 
proportion of households with feroale heads and households composed of unrelated 
individuals had increased, a trend continuing into the 1980 *s (Dyer, Hill, and 
Vedlitz, 1985). 

Although the proportion of men to women remained unchanged, the distribution 
of females and males within households had changed. The proportion of households 
with adults who would be missed in surveys by the Troldahl-Carter (T-C) method was 
predicted to increase with the continued growth of multiple-adult "primary" or 
unrelated adult households. 

Bryant was concerned that the T-C procedure led to over-representation of 
women in samples, a problem further complicated by the rising probability of 
young males to be away from home. Even increasing the number of call-backs was 
not improving the representation of younger men in surveys. 

The results of 19 major market and statewide surveys in 1973 and 1974 led 
Bryant to conclude that changing trends in household composition resulted in 3-, 
4-, and 5-adult households which were disproportionately female and 6- and 7-adult 
households which were disproportionately male. She reported no solution for the 
problem of unavailable young men; however, a violation of random selection in the 
T-C method could help to overcome the difficulty of female over-representation. 

One solution was an increase in the number of men who could be selected in 

"modal households," those with two adults, one male and one female. This was 

accomplished by eliminating the fourth T-C matrix, requiring a woman in this 
2 

kind of household. Although, mathematically, over-representation of males 
might be expected to result, this happened in only three of 12 additional 
markets tested, mainly because of the counter-balancing effect of the male 
nonresponse problem. 

However, a second modification of T-C appeared to be a better solution. 
This involved use of the fourth T-C matrix onlv half as often in surveys as th^ 



other three: thus the order of versions used was 1, 2, 3. 4. 1, 2. 3. This 
innovation decreased the probability of male selection from the estimated 55.0^ 
male/45. 0^ female for the first modification to an estimated 48.4% male/51. 65fe 
female ratio for the second modification. Bryant preferred the second for being 
closer to the 1970 census figaren of 47.4% male and being less likely to be 
affected by substitution of women for not-at-home men. (Kish also had reported 
problems with undar-representation of men in his 1949 article because of male 
nonresponse and unauthorized interviewer substitutions.) 

Both of these methods were within the 95% confidv.nce range of sampling 
error for the samples studied, although some bias might be systematic. The 
second method appeared to be less biased and more representative of the general 
population on demographics and at least one behavior. 
The Czala-Blair-Sebestik Modification 

No researchers appear to have published any comparisons of the most-used 
methods, Kish and T-C (or T-C plus Bryant — T-C-B) until Czaja, Blair, and 
Sebestik published their results testing Kish and two modifications of T-C-B in 
1982. Their Kish procedure involved eight tables, with four of the patterns 
having a relative frequency of use l/6th of the time and four patterns a 
frequency of l/12th. One T-C-B version asked for the number of men (T-C-B/men) 
and the other asked for the number of women (T-C-B/womeii) . 

Both T-C-B modifications had an advantage over Kish in sex representation. 
Despite similarities of response rates and some demographic chfracteristics, other 
dissimilar.Uies in results suggested caution in choice of these respondent 
selection methods when survey topics could be related to method biases, '^hese 
authors concluded that T-C-B/women had the most advantages, including better 
completion rates. Incidentally. T-C-B/women produced the highest proportion of 
listed telephone numbers, but the reason for this was not apparent. 

Czaia. et al. noted that Grovpp and Kahn (1979) used a Kish procedure (n ^ 




personal interview survey and a T-C procedure asking for women, noc men. in a 
telephone survey being compared with the personal one. obtaining results 
different from their own study. One reason may have been that Groves and Kahn 
used T-C with the question asking for women, and Czaja, et al. used T-C-B/women. 
which would lead to some differences in sex distribution (with T-C-B/women 
favoring greater representation of males). T-C and T-C-B have never been 
compared with each other in any puolished studies. Groves and Kahn thought 
that, compared to Klsh. the T-C/women version underestimated household size ana 
tended to under-represent young adults, particularly sons and daughters aged 18 
to 21. This led them to use Klsh procedures in telephone surveys as well a 
face-to-face surveys, resulting in improved respondent selection and response 
rates equivalent to the personal Interviews. However, the Groves and Kahn test 
of Klsh and T-C/women is not exactly comparable to Cza^a, et al, who used T-C- 
B/women, nor is it comparable to the surveys analyzed here because of 
differences between face-to-face and telephone survey modes. 
The Hagan-Collier "Alternative Non- invasive" P rocedure 

Another respondent selection metaiod was proposed by Hagan and Collier in a 
paper presented to the 1982 conference of the American Association for Public 
Opinion Research (published the next year). Intended to be a simplified. non- 
Invasive improvement on T-C or T-C-B because no household composition questions 
are asked, it uses four respondent selection forms distributed randomly to 
interviewers: (A) tne oldest man. (B) the youngest man. (C) the youngest woman, 
and (D) the oldest woman. Heeding Bryant's advice. Hagan and Collier used forms 
A. B. and C two times out of seven and form D one time out of seven. If the 
household had no such person requested, then the interviewer asked for the 
opposite sex of the same age group. 

They also ...mpared their innovation with T-C-B/men in a split-half national 
survey test, finding essentially no differences between the two snmplps in 
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completion rates and demographics. The new alternative method achieved a 
higher cooperation rate at the respondent selection phase, however. 

Lavrakas (1987) cautioned that interviewers can be confused because a woman 
In a one-person household qualifies either as the "youngest woman" or the 
"oldest woman." Respondents also can think the interviewer wants an old man or 
a young woman. Further, interviewers need to be instructed, for example, that 
the "youngest man" in a household can be an 80-year-old man. The non-invasive 
procedure lacks ability to sample »^ther adults in a household with more than two 
adults, like T~C-B and T-C. on whicii it is based. 

The "Next Birthdav'/"Most Recent Birthday" Method 

In 1980 Salmon and Nichols presented a different method, the birthday 
technique, in a paper for a Chicago meeting of the Midwest Association for 
Public Opinion Resciarch. later published in 1983. They described four 
respondent selection methods, including "next-birthday" (also "last-birthday" 
according to a proposed improvement). The birthday technique* requires neither 
enumeration of household members nor selection tables. The interviewer simply 
asks to speak to the adult household member with the next (or last) birthday. 
Their sample was small. 230 respondents in two Kentucky counties. Four 
selection techniques were compared: next-birthday. T-C. no selection, and 
male/female alternation. 

Salmon and Nichols added the caveat that, since data on individuals 
collected from a sample of household units often should be weighted, birthday- 
method survey data should be weighted by household size if demographic 
characteristics may vary by household size. Kish (1949) had noted such 
weighting could increase sampling error, but he wrote thav this error could be 
mitigated by the high concentration of the population in two-adult households. 
He was especially cautious, however, about weighting for household size if 
characteristics studied were related to household size. Most authors of the 
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surveys described here stated that data were weighted accoraing to the number of 
adults in the household. 

Theoretically, the birthday method allows all household members to have an 
equal probability of ^r'-lection, regardless of household size. No notentially 
intrusive questions c^me at the beginning of the interview, and the sample 
should be representative of the population at large. They explained: 

The next-birthday method is based on the assumption that the assignment 
of birthdates is a random process. When viewed as a single-stage 
sampling process, the next-birthday method will not produce a sample in 
which all household members have an equal probability of being selected. 
For example, suppose two people, one with a birthday of April 1 and the 
other with a birthday of April 2, reside within a selected household. If 
the interview takes place on a randomly selected day, the person with the 
birthday on April 1 has 364/365 probability of being selected while the 
person with the birthday on April 2 has only a 1/365 probability of being 
.selected. However, when the next-birthday method is viewed as the second 
stage of a two-stage sampling process, in which the firnt stage is 
completed when the person is born, the method produces an equal 
probability sample within a household, (p. 271) 

Howover, Lavrakas (1987) reported: "Theoretically these [birthday] methods 
represent true random selection, but in practice there is still some uncertainty 
whether this in fact results" (p. 97). He suggested researchers keep up with 
findings on validity by following published comparisons of the birthday method 
with other methods. 

O'Rourke and Blair (1983) compared the most-recent birthday method with the 
Kish procedure in a state-wide Illinois omnibus survey.'^ Demographic and 
substantive data were comparable between the two methods. They preferred the 
bil-thday method for ease of use, interviewer acceptance, and lower refusal rate. 

Tarnai, Rosa, and Scott (1987) also compared Kish and birthday methods, 
finding the birthday method surpassed Kish for increased interviewer ease of 
use, completions, and response rates, as well as for shorter interviews. The 
birthday method represented more multiple-adult households, and Kish was linked 
to more refusals to participate. Although both samples were comparable in 
demoeranhic renresentation of the t^onerf^l nopnlation, Tarnai. et al. concluded 
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that both procedures have somewhat different biases. 

Difficulties with both methods occurred more often in multiple-adult 
households. T.hese authors suggested Kish produces greater non-response error 
because of higher refusal rates. The birthday method is flawed because some 
respondents do not know all household members' birthdays, and the likelihood of 
knowing birthdays decreases as household size increases. 

In the Tarnai. et al. study. 2e% did not know birthdays for all household 
members, and another 29% of informants made incorrect respondent selections. In 
contrast. O'Rourke and Blair reported that 9* of their informants wore inaccurate 
in respondent choices and 1.1% did not know some of the birthdays for the 
household. They suspected, but could not calculate, a similar rate for their Kish 
sample. Sometimes informants were confused about the reason for the selection 
procedure and chose the person who would have the next birthday instead. 
Oldendick (1987) reported the lowest proportion of incorrect respondent 
selections. 2.3% (no mention was made of number of informants who did not know 
birthdays for all persons in the household). His Kish comparison sample also had 
selection errors, but he thought it impossible to calculate the rate. 

Oldendick (1987) compared Kish and last birthday methods in three 
surveys in Ohio, concluding that the birthday means of selection slightly 
reduced initial refusals. However. Kish-contacted respondents were more likely 
to be converted to completed interviews, so that overall completion rates were 
similar. Demographic characteristics also were similar except on race. The 
last birthday method represented more blacks, and the distribution of blacks was 
more similar to 1980 Census data than was the Kish sample. 

Substantive questions in Oldendick^s report showed only one statistically 
significant difference — on affirmative action ~ in a direction suggesting 
relationship to the greater representation of blacks in the birthday sample. 
However, neither Haran and Collier (1983) nor Zukin. et al nf>87) found ^nv 
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difference In race between their comparisons (birthday and an alternative to T- 
C-B and a non-random method, respectively). Czaja, et al (1982) found only 2% 
more blacks with Kish than with either T-C-B version. 

Incidentally, ali three of Oldendick's birthday samples over-represented 
persons in single-adult households and females, a non-significant but consistent 
difference. However, O'Rcurke and Blair (1983) and Tarnai, et al (1987) found 
more multiple-adult households in their birthday samples than in their 
comparison Kish seunples. 

OldendAck explained differences between his study and similar comparisons 
of Kish and birthday methods: populations were different, callbacks were 
pursued mort vigorously in his survey, more refusals were re-contacted to 
attempt conversions to completions, and his irterviewing service had had much 
more experience with the birthday method than other interview firms may have 
had. Another difference may have been sample size. 

Even though Oldendlck reported fewer differences between birthday and Kish 
procedures, his survey organization switched from Kish to birthday because the 
birthday method was easier to administer, produced demographically equivalent 
samples, led to similar substantive results, and improved response rates. 
Increased representation of blacks may have been a strong factor also. 

Zukin, Carter, and Schulman (1987) compared the last birthday method and a 
non-random method, interviewing the first eligible respondent contacted, in two 
matched cross-sectional samples in New Jersey in 1986. No substantial 
differences were found between the two samples for representativeness of "basic 
demographics" (education, age, race, income) **basic political orientations" 
(partisanship, ideology), or "substantive questions in a number of policy areas." 

Eighty percent of interviews in their last-birthday sample were completed 
with the respondent who answered the phone, even after screening for the 
designated respondent. fO'Rourke and Blair reported a rate of about two- 
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thirds.) Zukin, et ai, argued for use of the non-probability method because of 
a significantly higi. ^ refusal rate (size not reported) and greater 
administration expense with the birthday method. 

ThelT* study apparently included much respondent substitution, but how much 
was not reported. If the person with the most recent birthday was unavailable 
for the duration of the survey, the person with the next most recent birthday 
was chosen. 

Both the Zukin, et al, samples of about 400 respondents used sex quotas to 
obtain more males after the first 200 Interviews. This characteristic suggests 
that they should have compared four groups, not two: the birthday and the non- 
random samples by the first and second sets of 200 interviews. 
Assessment of the Seven Comparative Studies 

In Table 2, the most relevant comparisons are within studies, not across 
studies, since the bases differ among them. On the whole, the overall highest 
completion rates occur for the birthday, no-selection, and T-C-B/women 
procedures, when these are compared with other procedures. The "alternative" 
method did just slightly better than T-C-B/men, and T-C-B/men had the lowest 
completion rate when compared with T-C-B/women and Kish methods. Kish 
completion rates are respectable but not quite as good as other methods with 
which Kish has been compared. Only one study used a "pure" T-C method (Salmon 
t,nd Nichols, 1983). Other variables, however, can account for differences among 
studies, including variation in research staff experience and training, region, 
size of community, and rijjorousness of control over all phases of the surveys. 

It is difficult to tell which method might be "best" because many 
potential combinations have not been tested, as Table 1 shows. Seven studies 
are too limited a number to allow firm conclucions. Some results vary somewhat 
among those methods which have been compared in more than one study. Only 
Oldendick (1987) built reolication into his design. 
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The seven studies taken together indicate the non-probability no-selection 
method is the least representative, as would be expected, and that the birthday 
method, T-C-B/men, T-C-B/women, and the Collier-Hagan alternate method result in 
the most representative samples. Kish and Troldahl-Carter methods theoretically 
should be the best, but they seem to have a bias toward disproportionate 
representation of women, smaller households, and lower SES characteristics. 
This bias, however, could counterbalance the high-SES bias in random-digit- 
dialing surveys. 

The four methods which seem to be most representative seem to be the best 
candidates for continued testing: Hagan-Collier, birthday, T-C-B/men, and T-C- 
B/women. Each of these methods has drawbacks. The Hagan-Collier method tends 
somewhat to under-represent multiple households, and the birthday method has the 
advantage of representing such households better. It is difficult to tell how 
representative T-C-B/men and T-C-B/women are in comparison with each other from 
reports with household size data (Czaja, et al , 1982; Hagan and Collier, 1982), 
Only a comparison with census data can show which method leads to results most 
in line with population household characteristics — ascuming that up-to-date 
census data are accurate and available. 

The point is often made that the relevant comparisons are the data from 
each comparative study. However, it is difficult to assess surveys on 
respondent selection without knowing comparable census data as well. If one 
sample is relatively younger and better educated, and another one is relatively 
older and less educated, it is difficult to determine which is more 
representative unless population characteristics are compared to each sample. 
Because random-digit-dialing samples actually represent telephone lines rather 
than households, such samples tend to over-represent older persons, whites, non- 
Hlspanics, and those with high incomes (Schmidley, 1986). It is likely that ROD 
samoles. therefore, should be weighted according to distribution of these 
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characteristic? in the population, especially if these characteristics are 
related to topics studied. (Czaja. Blair, and Sebestik, 1982. appear to be the 
only investigators among the seven comparative studies who weighted data 
according to number of telephone lines.) 

The birthday method's own biases present some problems, sometimes severe 
ones. Only O'Rourke and Blair have looked for ~ and found ~ a skew of more 
respondents* birthdays in months occurring closest to interviewing. In 
addition, respondents may lie about who in the household has the next birthday, 
either because they are too lazy to call that person to the phone, or possibly 
because they themselves want to be interviewed. 

The proportion of informants who actually do not know birthdays for the 
household ranges widely so far, from 2.3% to almost 30%. Degree of substitution 
in birthday-method samples impedes comparability. Most studies using birthday 
methods have done some substitution if the appropriate person was not available 
for the duration of the study. 

Little, if any, evidence since Bryant's article shows what sex quotas do to 
probability samples. The Zukin, et al, survey offers the opportunity to look 
into this issue. Such a comparison is important because proportionate 
representation of males and females is a frequent survey problem. Is it better 
to build in a preference for males in a selection method before interviewing, or 
is it better to use a quota for males and females, when an imbalance appears to 
be increasing? Or, should random selection never be violated,^ and corrections 
for sex imbalances made only by weighting data after the survey? 

The problem of over-representation of omen may not be substantially 
related to any particular method. First, women tend to answer the phone more 
often than men do — about two-thirds of the time (Dillman, 1978). Answering 
the phone may be part of women's roles in the home. Second, women are more 
likelv to be at home, and men. esneciallv youn^ men. are less likely to be at 
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home. Third, the number of households headed by a woman ~ either family or 
non-family households, including female one-person households, has been 
increasing (Lavrakas. 1987; Dyer. Hill, and Vedlitz. 1985). 

More work should be done on estimating the number of respondents selected 
incorrectly by informants, not only for the birthday method, but also for other 
methods. Further, though several studies record little distortion in variables 
besides demographics, such as attitude variables, much work is needed also on 
differences in bivariate and multivariate distributions. 

More national surveys are needed. Six of the seven surveys reviewed 
were regional, state-wide, or smaller areas. Only Hagan and Collier had a 
national sample (see Table 3). Most surveys have the effects of urbanism 
mixed in among other influences on variation. 

Few researchers give the wording of their respondent selection questions. 
Even this wording could Influence results and survey comparability. Much more 
replication of comparative research designs is desirable. Further, amount of 
interviewer experience with different methods should be considered a variable if 
it is not possible to use interviewers or interview services with comparable 
experience in the methods tested. Another possibility is to have all 
interviewers administer all methods tested. 

Other information often omitted from research reports includes: topics 
studied, interview length and dates, how completion rates were calculated, 
number of callbacks, or degree of respondent substitution. If future research 
reports better addressed these points, comparisons of studies would be 
facilitated. It is surprising that more comparisons have not been made of 
respondent selection techniques, especially for research using random-digit- 
dialing samples. The available information on such comparisons is not adequate 
for researchers who need to evaluate the most effective strategies for their 
research projects. 
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Notes 



1. According to Paisley's and Parker *s Table 2 on p. 435. one example of 
question wording, from the third of the 60 versions, is: 

First, would you tell me how many persons in your home are over 18 and no 
longer in school ~ (CIRCLE NUMBER IN FIRST COLUMN BELOW) 

How many of them are women — (CIRCLE NUMBER IN SECOND COLUMN BELOW)... 

Then, according to my instructions, I»m supposed to interview the (PERSON 
INDICATED ABOVE). Is (HE, SHE) available now ~ 

2. For a number of reasons informants (those persons who answer the phone 
first) get hurt feelings when told the interviewer wants to speak to another 
person (the chosen respondent). Lavrakas (1987) suggested a standardized 
response for interviewers: 

I have a sheet here that tells me the one person in youi household whom I 
can interview. Sometimes this sheet picks a woman, and ' ther times it 
picks a man. Only by using this method to select people to interview in 
each household we contact will we end our survey with a fair and proper 
balance of females and males, (p. 93) 

3. The question wording is: "May I please speak to the [NAME, e.g., 
•youngest man?']" (p. 549j. Lavrakas (1987) pointed out that it is a good 
idea to add words stating that the person wanted is not the one who happens 
to be at home at the time but is instead the one who lives in the household. 
He also gave several wording examples, such as: "For this survey, I need to 
speak with the youngest adult male in your household over the age of 17, if 
there is one." [IF NONE:] "Then may I please speak with the youngest adult 
female?" 

4. The wording of the Kish question in O^Rourke and Blair (1983) is: 

In order to determine whom to interview, I need to list household members 
who currently live in this household, i just need their age and sex — 
not their names — and how they are related to you. Let's start with 
you. How old are you? Now, can you tell me the sex and age of all other 
current household members and how they are related to you? (RECORD ON 
CHART.) Now I'm going to number the people in your household to 

determine whom I should interview. The total number of persons is , 

so I am to interview person # , who is (RELATIONSHIP), (p. 430) 

The wording of the birthday question is: 

In order to determine whom to interview, could you tell me. of the people 
who currently live in your household who are 18 or older, who had the 
»ost recent birthday ? (IF DON'T KNOW ALL BIRTHDAYS): Of the ones that 
you do know, who had the most recent birthday? 
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5. The wording of the birthday question in Oldendick's report is slightly 
different from that of O'Rourke and Blair, according to their communications 
lo him about respondent reactions: 

In order to determine who to interview, coula you tell me. of the people 
who currently live in your household who are 18 or olde*.^ — including 
yourself — who had the most recent birthday? I don't mean who is the 
youngest adult, but rather, who had the most recent birthday? (IF 
INFORMANT DOES NOT KNOW ALL BIRTHDAYS:! Of the ones you do know, who had 
the most recent birthday? (p. 2) 

6. This may vary by region. It is possible that the birthday method works 
better in some regions than in others. (Telephone communication from 
Charles T. Salmon in November 1984.) 
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Table 1^ Respondent Selection Methods Compared in Research. 



Authors and Dates of Articles or Papers 



Czaja, Hagan & Oldendick, O'Rourke Salmon & Tarnai Zukin. 

Blair, & Collier, 1987 & Plair, Nichols, Rosa, & Carter, & 

Sebestik, 1983 1983 1983 Scott, Schulraan 

1932 1987 1987 



Method 

No-Selection 
(Non- random) 

Male/Jemale 
Alternation 
(Non-random) 

Kish 

Troldahl/ 
Carter (T-C) 



Troldahl/ 
Carter 
Bryant 
(T-C-B/men) 



X 



T-C-B 
Version 
Asking for 
Women 

Next- 
Birthday/ 
Most Recent 
Birthday 



Non-invasive 
Alternative 
to Troldahl/ 
Carter 



Table 2^ Refusals. Respondent Substitution, and Comparability of Research. 



Authors and Dates of Articles or Papers 



Czaja. Hagan & 01u«»ndick, O'Rourke Salmon & Tarnai Zukin. 

Blair, & Collier, 1987 & Blair, Nichols. Rosa, & Carter, & 

Sebestik, 1983 1983 1983 Scott, Schulman 

1982 1937 1937 



Comple- 
tion 
rate 



Tell how 
refusal 
rate cal- 
culated? 



Kish: 
68.6% 

TCB/women: 
70.3% 

TCB/men: 
64.6% 



Yes 



TCB/men : ' 
64.6% 

Alterna- 
tive 
method: 
64.9% 



Yes 



Kish 1: 
82.6% 
B'day 1: 
82.8% 

Kish 2: 
83.4% 
B'day 2: 
81.0% 

Kish 3: 
82.2% 
B'day 3: 
84.0% 

Yes 



Kish: 
61% 

B'day: 
67% 



Yes 



No-selec- Kish: 
tion: 70% 
69% 

B'day: 
M/F alter- 77% 
nation: 
43% 

Next b'day: 
695^ 

Troldahl/ 

Carter: 

41% 

No Yes 



No-selec- 
tion: 
80% 

B'day: 
60% 



No 



Question- Yes Yes 

naires 

identical 

each 

version? 

Respondent Apparent- No^ 
substitu- ly not 
tions 
allowed? 



Yes 



No 



Yes 



Apparent- 
ly not 



Yes 



Yes 



Not given Yes 



Yes 



Yes' 



Figures based on recalculation of original authors* data, excluding nonhouseholds 
l^and people ineligible by the survey^s definition. 

^The major portion of refusals occurred at the respondent selection procedure. 
Author<^. do not give exact figures for refusal rates; therefore, these figures are 
^guesses, based on wording of Zukin, et al , paper. 

^This survey had a part 2 mail questionnaire portion. These data are for part 1. 

If selected respondent was unavailable, any "responsible adult" was interviewed 
^about household demographics. Questionnaire mailed to designated respondent. 

If respondent was not available for the duration of the study, interviewers took 
^another Individual in the same household with the next most recent birthday. 

Information supplied by John Tarnai in phone call. May 11. 1988. 
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Table 3^ Survey and Sample Characteristics. 



Authors and Dates of Articles or Papers 





Czaja, 


Hagan & 


Oldendick, 


O'Rourke 


Salmon & 


Tarnai f-? 


Zukin, 




Blair, & 


Collier, 


1987 


& Blair, 


Nichols , 


Rosa, &^ 


Carter, & 






1 Qfi') 
1 ^DO 




1983 


1983 


Scott. 


Schulman 




1982 










1987 


1987 


Survey 


Not given 


Not given 


June 1983 


June 19r2 


Not given 


May 1986^ 


2 wave s 


dates 






Oct. 1983 
Nov. 1984^ 






July '86 














Location 


Cook 


National 


2 surveys 


State of 


Two 


Four 


Two 




County, 


sample 


in Ohio, 


Illinois 


counties 


counties 


separate 




111. 




1 in 

Cincinnati 




in 

Kentucky 


in Wash- 
ington St 


surveys 
. in N.J. 


Sample 


Kish: 


TCB/men: 


Kish 1: 


Kish: 


No-selec- 


Kish: 


No-selec- 


size 


2,650 


1,237 


596 

B'day 1: 


499 


tion: 
72 


877 


tion: 
399 




I Ld/ women : 


Alterna- 


588 


B * day : 




B'day: 






1,322 


tive 




557 


M/F alter- 


950 


B ' day : 






method 


Kish 2: 




nation: 




400 




TCB/men: 


1,236 


487 




46 






1,322 




B'day 2: 
485 

Kish 3: 
723 

B'day 3: 
739 




Next b'day 
69 

T-C 
43 






Sample 


Random- 


Random- 


Random- 


Chicago 


Systematic RDD, 


Random- 


type 


digit- 


digit- 


digit- 


SMS A: 


sample of 


stratified digit- 




dialing 


dialing 


dialing 


directory- 


numbers 


by county 


dialing 










based ROD; 


from phone 












Other : 


book 














systematic 
















phone book 
















sample 








No. of 


Up to 10 


Up to 4 


Up to 6 


Up to 10 


Up to 3 


Up to 4^ 


Up to 4 


calls 












Survey 


Issues 


Consumer 


First 2: 


Road condi- 


Politi- 


Socioec. 


Political 


topics 


related 


topics 


political , 


tions, city 


cal 


well- 


public 




to per- 




economic; 


government , 


atti- 


being. 


policy 




sonal 




3rd metro: 


education. 


tudes 


job ^ 


issues 




safety, etc. 


local 


energy, etc. 


training 





j^Information supplied by authc . 
Information supoli^^d bv John Tarnai in phone call. May 11 
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